Feature: Add pandoc for typst future format conversion #6
Conversation
merge master branch into this branch.
merge master branch
|
@Wellshh is attempting to deploy a commit to the alephpi's projects Team on Vercel. A member of the Team first needs to authorize it. |
✅ Deploy Preview for texocr ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Wow thank you for your PR, that was a big amount of work. However, I'm not intended to maintain a server for this project at the moment(especially for this tiny conversion feature), that's why I made the Texo model running entirely in the browser and take advantage of the free SSG at Netlify. It would be better if you can make pandoc run entirely in the frontend (maybe via wasm? But still what would be the wasm size of pandoc? After all you know the onnxruntime-wasm is ~20MB, if we have something way more bigger, it won't be reasonable to deliver this. But if we only need the latex2typst subset functionality of pandoc , maybe it won't be very large) I don't mean to waste your valuable effort. Maybe you should have opened a feature request before submit the PR so we could swap opinions on the solution. Of course, if you can afford a server, don't hesitate to deploy this solution on your own. |
|
Thank you so much for the quick and incredibly detailed feedback! I really admired ur work so that’s why I’m submitted this PR. I didn’t know much about the WASM thing, maybe I should dig into it later. |
|
I happen to check pandoc wasm solutions just now, like here, however according to this PR, the memory usage is kind of insane, so I doubt its feasibility. Another decrepated solution is here, but the conversion result in the demo page is not very satisfying(perhaps due to the old version) |
2 participants
I have some experience in latex2typst conversion, where using pandoc is a more robust and scalable solution (I have tried tex2typst as well).
However, this means when deploying to production server, may need to download the pandoc binary. For development, I have written the docker file.
This pull request introduces a Docker-based development and testing workflow, adds a Pandoc-powered LaTeX-to-Typst conversion service with concurrency control, and implements comprehensive automated tests for both client and server conversion logic. The changes also include new configuration files for Docker and testing, updates to dependencies, and improvements to the build pipeline.
Dockerization and CI/CD improvements:
Dockerfileanddocker-compose.yamlto enable containerized development, testing, and production builds for the app, including multi-stage builds for development, test, and production environments. (Dockerfile,docker-compose.yaml) [1] [2].github/workflows/ci.ymlto build and run tests, linting, and typechecking inside Docker containers for improved environment consistency. (.github/workflows/ci.yml) [1] [2].dockerignoreto exclude unnecessary files from Docker build context. (.dockerignore)LaTeX-to-Typst conversion service:
server/api/convert/typst.post.ts) and a Pandoc process manager (server/utils/pandocManager.ts) to convert LaTeX to Typst using Pandoc, with concurrency limits and timeout handling. (server/api/convert/typst.post.ts,server/utils/pandocManager.ts) [1] [2]app/composables/textProcessor.tsto support both local and server-based conversion, with caching and runtime detection. (app/composables/textProcessor.ts,app/composables/types/pandoc.ts) [1] [2]Automated testing enhancements:
vitest.config.ts), test scripts topackage.json, and dependencies for unit and Nuxt environment tests. (vitest.config.ts,package.json) [1] [2] [3]test/unit/pandocManager.spec.ts).test/nuxt/textProcessor.cache.spec.ts).Miscellaneous improvements:
nuxt.config.ts. (package.json,nuxt.config.ts) [1] [2]app/pages/ocr.vueto use async conversion for clipboard copying. (app/pages/ocr.vue)